• The final type of design we will discuss today is the waitlist design, also known as the stepped-wedge design
• Their scientific value comes from their ability to track treatment effects among several subjects as they play out over time
• Waitlists play a diplomatic role because they overcome the problem of withholding treatment from a control group (remember our ethics discussion?)
• In this design, every subject is treated eventually; random assignment determines when they receive treatment
• It is a hybrid between a within-subject and a between-subject design, and it is widely used in public health interventions (e.g., vaccine rollouts)

Source: Hemming et al (2014)

• The immediate treatment effect is \(Y_{01} - Y_{00}\), that is, the effect of being treated in the current period but not in the preceding period
• We just need to take the numbers from the tables and apply inverse probability weighting again

\[ \begin{aligned} \widehat{E}[Y_{01} - Y_{00}] &= \frac{\frac{7 + 10}{0.25} + \frac{2 + 1}{0.25} + \frac{7 + 10}{0.25}}{\frac{2}{0.25} + \frac{2}{0.25} + \frac{2}{0.25}} \\ &- \frac{\frac{7 + 1 + 4 + 4 + 3 + 2 + 3}{0.75} + \frac{5 + 3 + 3 + 3}{0.50} + \frac{3 + 4}{0.25}}{\frac{6}{0.75} + \frac{4}{0.50} + \frac{2}{0.25}} = 2.72. \end{aligned} \]

• Finally, we will estimate the cumulative effect of TV advertising, which is \(Y_{11} - Y_{00}\), that is, the effect of being treated in both the preceding and current periods
• We do the same thing again, but now we consider the \(Y_{11}\) potential outcomes
• However, we must restrict our attention to the second and third weeks, because this type of treatment cannot occur in the first week

\[ \begin{aligned} \widehat{E}[Y_{11} - Y_{00}] &= \frac{\frac{9 + 8}{0.25} + \frac{4 + 10 + 10 + 3}{0.50}}{\frac{2}{0.25} + \frac{4}{0.50}} \\ &- \frac{\frac{5 + 3 + 3 + 3}{0.50} + \frac{3 + 4}{0.25}}{\frac{4}{0.50} + \frac{2}{0.25}} = 4.13. \end{aligned} \]

• Can you easily manipulate the network structure?
• Not always, but you can use random assignment to create different network structures
• The author created an online health community with 1528 participants recruited from internet forums
• All participants were anonymous and were partnered with “health buddies” to share information about health behaviours
• Participants made decisions about whether or not to adopt a health behaviour based on the adoption patterns of their health buddies
• The health behavior used for this study was the decision to register for an Internet-based health forum
• What do you think of this design so far?

• The author assigned participants to one of two network structures:
  • Clustered network with highly connected participants
  • Random network with weak ties between participants
• Treatment was assigned before the experiment started and participants were not aware of their treatment assignment
• The author claim the design is better than observational studies because it eliminates confounding variables such as
  • Personality traits
  • Geographic location
  • Advertising and pricing
  • Changes in the networks over time
• Can you do the same with massive data from social media?

• Diffusion was initiated by selecting a random “seed node”, which sent signals to its network neighbours encouraging them to adopt a health-related behaviour
• Every time a participant adopted the behaviour (i.e., registered for the health forum), messages were sent to their health buddies inviting them to adopt
• If a participant had multiple health buddies who adopted the behaviour, then they would receive multiple signals, one from each neighbour
• The more neighbors who adopted, the more reinforcing signals a participant received
• The author measured the fraction of participants who adopted the behaviour

• Cumulative distribution of forum visits over time. The lower, the higher the likelihood of checking the forum

• The author concludes that clustered networks are more effective than random networks in spreading behaviour
• But there are some limitations to this study, too
• Can you think of any?
• Can you think of policy implications?

• Multilevel experiment across 56 US middle schools (24,191 students)
  • Randomised at school level (treatment vs. control)
  • Randomised at student level within treatment schools (seed students)
• Comprehensive measurement of school social networks before intervention
• “Who do you choose to spend time with?” - measures attention in network
• Identified “social referents” - top 10% most connected students in each school
• Intervention: Anti-conflict campaign driven by seed students
• Encouraged public stance against conflict
• Seed students (20-32 per school in treatment schools) were selected to participate
• Intervention activities: creating slogans, posters, wristbands

• Grassroots campaign approach
• Seed students identified school-specific conflict behaviours
• Developed slogans and posters
• Distributed wristbands as rewards for positive behaviour
• No top-down, adult-defined problems: focused on student perspectives
• Standardised procedures with trained facilitators using scripts and activity guides
• High attendance at meetings (average >55% of invited students)
• Made meetings engaging: snacks, hands-on activities, technology

• Outcomes measured subjectively and administratively
• Subjective: student-reported norms about conflict (pre/post surveys)
• Descriptive norms: how many students participate in conflict?
• Prescriptive norms: how many students disapprove of conflict?
• Administrative: school disciplinary records of peer conflict events (across the year)
• Focus on disciplinary events as a key behavioural outcome
• Arguably less susceptible to reporting bias than self-reported conflict

• Significant reduction in disciplinary reports in treatment schools ~30% reduction overall
• Effect stronger when seed groups contained more “social referent” students
• Increased talking about conflict reduction and wearing wristbands in treatment schools
• No average differences in social norms between treatment and control schools initially
• Social referent seeds were more influential than other seed students
  • 20% social referent seeds -> 60% reduction in disciplinary events
• Students exposed to social referent seeds more likely to:
  • Report friends discussing conflict reduction
  • Perceive stronger anti-conflict norms
  • Wear wristbands
• No peer-to-peer influence effect on discipline directly (climate-level effect stronger)

• Intervention shifted school climate - widespread behavioural pattern
• Social referents key to norm and behaviour change
• Outsized influence due to attention from peers
• Not just structural position, but also traits and experiences
• Norms findings nuanced:
  • No overall shift in norms between schools
  • But within treatment schools, norms did shift, especially via social referents
  • Possible “signal” effect of intervention - increased attention to conflict, leading to revised norm evaluations

• Peer influence interventions can be effective in reducing school conflict
• Importance of social referents in changing behaviour
• Target these individuals in interventions for greater impact
• Highlights the usefulness of student-led initiatives
• Grassroots approach can be more effective than top-down
• Methodological contribution:
• Demonstrates rigorous social network experiment in real-world setting
• Multilevel randomisation for causal inference in networks

• What are some potential limitations of this study?
• Generalisability to other contexts? (different types of conflict, schools, cultures)
• Long-term effects?
• Mechanism of change fully understood?
• Reliance on disciplinary records - are these fully objective?
• Ethical considerations of network interventions? (manipulation of social structures?)
• What about the students who were not selected as “seeds” but were still part of the network? Were they affected negatively?

• Large-scale field experiment in New Haven, Connecticut (1998 election)
  • ~30,000 registered voters
  • Random assignment at household level to different GOTV treatments & control
• 2x2x4 Factorial Design:
  • Personal Canvassing
  • Telephone Calls
  • Direct Mail (None, One, Two, or Three mailings)
• Non-partisan messages used for all treatments
  • Civic duty, close election, neighbourhood solidarity appeals
• Outcome: Validated voter turnout from official records

• Personal Canvassing:
  • Paid canvassers (grad students) in pairs, weekends before election
  • Targeted specific households, not entire streets
  • Varied message appeals (civic duty, close election, neighbourhood solidarity)
  • Contact rate: ~28%
• Direct Mail:
  • Postcards designed by political consulting firm
  • Varied number of mailings (1, 2, or 3) and message appeals
• Telephone Calls:
  • Out-of-state telemarketing firm, calls in days before election
  • Scripted calls, civic duty and close election appeals only
  • Contact rate: ~32% (reached household)

• Issue: Not everyone in treatment groups actually received treatment
  • Simply comparing contacted vs. non-contacted inflates effect
• Solution: Two-stage least squares (2SLS) regression using intent-to-treat as instrument
  • Instrument: Random assignment to treatment group (yes/no)
  • Endogenous variable: Actual contact (yes/no)
• Rationale: Random assignment predicts contact, but is uncorrelated with other reasons for turnout (no direct effect on turnout except through treatment)
• Allows for causal estimation of treatment effect accounting for non-compliance (non-contact)

• Personal Canvassing Highly Effective:
  • +8.7 percentage points increase in turnout (2SLS estimate)
  • ~9.8 points in multivariate model
  • Most effective GOTV method tested
• Direct Mail Shows Small Effect:
  • +0.6 points per mailing (multivariate model)
  • Small but statistically significant
  • Cumulative effect of 3 mailings ~ 2.5 points (probit)
• Telephone Calls Ineffective:
  • No significant effect, slight negative effect in some models
  • Despite professional firm and mirrored messaging

• Message Type Mattered Little for Personal Canvassing:
  • Civic duty, close election, neighbourhood solidarity appeals similarly effective (though close election slightly higher)
  • No significant interaction effects between treatments
• Cost-Effectiveness Analysis:
  • Personal canvassing: ~$8 per additional vote
  • Direct mail (3 mailings): ~$40 per additional vote
  • Face-to-face mobilisation significantly more cost-effective

• Supports Hypothesis: Personal canvassing > impersonal methods for turnout
  • Decline in face-to-face mobilisation may contribute to turnout decline
• Unexpected Finding: Ineffectiveness of telephone calls
  • Despite professional execution & mirroring canvassing messages
  • Possible reasons: Routinized/scripted nature, out-of-state callers
• Mechanism of Personal Canvassing Effect?
  • Speculation: Salience, memorability, connection, urgency… (future research needed)
• Limitations: Non-partisan messages, specific election context, short-term effects

• Personal canvassing is a valuable GOTV tool
  • More effective & cost-effective than mail/phone
• Challenges conventional wisdom? - Heavy reliance on mail/phone in modern campaigns
• Highlights importance of personal connection in political mobilisation
• Methodological Contribution:
  • Rigorous field experiment in political science
  • Addresses non-compliance with IV approach
• Future Research: Explore mechanisms, partisan vs. non-partisan, long-term effects, interactions of methods

• What are potential limitations of this text?
  • Generalisability to other elections/contexts?
  • Focus on turnout only - what about vote choice/persuasion?
  • Short-term effects only - sustained impact?
  • Specific message content - would different messages work better for mail/phone?
  • Ethical considerations of GOTV experiments? (Manipulation of voters?)
  • Hawthorne effect of canvassing? (People vote more because of the attention, not just the message)